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Portable  Wearable  Immersive  Virtual  Environment  Systems 


Virtual  environments  (VE)  have  become  an  invaluable  tool  for 
research,  development,  training,  healthcare,  commerce,  communication, 
and  education,  as  well  as  a  medium  for  entertainment.  Immersive  virtual 
environments  (IVE)  which  incorporate  a  wearable  rendering  system  and  a 
head  mounted  display  (HMD)  allow  users  to  explore  simulated 
environments  by  naturally  walking,  turning,  and  looking  (Fig  1.).  Such 
natural  navigation  and  the  associated  multisensory  stimulation  have  been 
shown  to  provide  a  greater  sense  of  realism  to  users  (Ruddle  et  al.  2011, 
Usoh  et  al.  1999,  Slater  et  al.  1995a).  Indeed,  one  of  the  strengths  of  VEs 
is  the  ability  to  present  multi-modal  sensory  information  to  users  to 
holistically  simulate  the  experience  of  being  present  or  immersed  in  the 
synthetic,  virtual  reality.  Rather  than  merely  presenting  optic  flow, 
compelling  visual  information  is  augmented  with  some  combination  of 
positional  auditory  cues,  proprioceptive,  kinesthetic,  and  inertial  feedback 
about  one’s  movement. 

With  support  from  the  Army  Research  Office  (ARO)  and  the 
National  Science  Foundation  (NSF),  the  Pis  have  spent  the  last  several 
years  developing  wearable  IVE  systems,  using  backpack-mounted 
rendering  computers,  HMDs,  and  body-worn  motion  sensors.  This  research 
can  be  separated  into  two  major  areas  of  focus.  This  first  is  the  study 
redirected  walking  (RDW)  techniques  (funded  primarily  by  ARO).  By 
imperceptibly  rotating  the  virtual  world  about  the  user,  RDW  makes  it 
possible  for  immersed  users  to  avoid  tracking  area  boundaries  while  exploring 
virtual  worlds  which  can  be  unlimited  in  extent.  The  second  area  of  focus 
involves  portable  position  tracking  systems.  The  goal  of  this  work  is  to  produce 
systems  with  require  little  or  no  setup  prior  to  and  can  be  used  in  and  moved 
to  any  location.  Through  this  research,  systems  which  are  capable  of  providing 
both  the  smooth  high  frequency  position  updates  necessary  to  an  effective 
locomotion  interface  and  the  absolute  position  necessary  for  RDW  and 
obstacle  avoidance  have  been  created.  To  date  the  research  has  spawned 
one  book  chapter  (Waller  &  Hodgson  2013),  seven  peer-reviewed  publications 
(Bachmann  et  al.  2012a;  Zmuda  et  al.  2012;  Hodgson  &  Bachmann  2013; 
Hodgson  et  al.  2011;  Yun  et  al.  2012;  Bachmann  et  al.  2012b;  Yun  et  al.  2011), 
three  invited  talks,  and  an  additional  six  manuscripts  that  are  currently  in 
preparation  or  under  peer  review.  The  funding  has  also  resulted  in  the 
creation  of  1  1/2  full  time  jobs,  has  provided  financial  support  for  five  graduate 
students  and  one  postdoctoral  fellow,  and  has  spawned  talks  with  seven 
regional,  national,  and  international  industry  partners  about  commercialization 
opportunities  or  applications  of  the  hardware  and  software  developed  under 
this  grant.  Finally,  the  Pis  are  in  talks  with  a  local  high  school  about  taking  the 
portable  IVE  system  to  their  gym  to  exhibit  to  students  and  support  a  curricular 
project. 


Fig  1 .  Portable  wearable 
IVE  system  in  an 
outdoor  environment. 


Unlike  traditional  HMD-based  systems,  users  of  portable  IVE  systems  Fj  2  Portab|e 
which  incorporate  a  wearable  rendering  systems  are  completely  untethered  wearable  IVE  system  in 

and  can  roam  freely  over  a  wide  area.  Proprioceptive,  vestibular,  and  efferent  use  jn  an  jnc|00r 

sensory  information  for  virtual  walking  in  a  forward,  sideways,  or  backward  environment, 
direction  or  potentially  running  are  nearly  perfect,  because  users  are  in  fact 

actually  moving  naturally  in  the  physical  world.  The  latest  iteration  of  this  system  is  free  of  any  fixed 
infrastructure  and  can  easily  be  used  in  any  large  open  area  such  as  a  parking  lot,  field,  hanger, 
gymnasium,  or  conceivably  even  a  remote  desert.  It  is  the  opinion  of  Pis'  that  using  natural  walking  as  the 
locomotion  interface  provides  the  best  and  most  veridical  sensory  information  to  users  as  they  move,  and 
is  the  most  intuitive  and  user-friendly  method  of  navigation  -  no  training  is  ever  required. 

Historically,  the  tradeoff  of  such  a  natural  interface  was  that  the  range  of  user  motion  was 
severely  limited.  If  one  had  a  10m  x  10m  lab,  for  example,  a  user  could  not  safely  walk  more  than  4m 
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from  the  lab’s  center.  However,  recent  advances  by  the  Pis’  and  others  (described  below)  have  made  it 
possible  to  overcome  this  using  a  technique  known  as  Redirected  Walking  (Razzaque  2005),  and  to 
enable  users  to  walk  naturally  through  VE's  of  unlimited  extent.  Likewise  the  development  of  portable 
position  tracking  technologies  which  require  no  infrastructure  and  little  setup  has  made  it  possible  to 
transport  such  a  system  in  the  trunk  of  a  small  car  to  much  larger,  ad-hoc  tracking  areas. 

Figures  1  and  2  depict  3rd-generation  wearable  IVE  systems  built  by  the  Pis.  Common 
components  in  both  systems  include  a  HMD  to  which  an  inertial/magnetic  orientation  has  been  attached, 
a  backpack  mounted  laptop  used  as  a  wearable  rendering  unit,  and  a  backpack  mounted  video  control 
unit  and  battery  pack.  In  both  figures,  foot  mounted  inertial/magnetic  sensors  modules  are  attached  to  the 
feet  for  the  purpose  of  infrastructure  free  position  tracking.  In  the  outdoor  configuration  depicted  in  Fig.  1, 
absolute  position  fixes  are  obtained  on  a  periodic  basis  using  the  mast  mounted  GPS  antennae.  Fig.  2 
depicts  a  system  use  in  an  indoor  environment.  Again  foot  mounted  inertial/magnetic  sensor  modules  are 
used  to  obtain  smooth  high  frequency  position  updates.  Absolutes  position  fixes  used  for  drift  correction 
are  obtained  from  an  ultrasonic  TDOA  (time  difference  of  arrival)  system.  More  recent  work  by  the 
principle  investigators  has  indicated  that  both  the  TDOA  sensors  and  the  foot  mounted  could  be  replaced 
by  a  stationary  or  backpack  mounted  scanning  laser  range  finder. 

Redirected  Walking 

With  traditional  immersive  VE  systems,  the  scale  of  a 
virtual  simulation  is  typically  limited  to  the  size  of  the  tracking 
area.  RDW,  a  technique  first  examined  by  Razzaque  (2005), 
overcomes  the  physical  limitations  of  a  tracking  area  by 
imperceptibly  rotating  the  virtual  scene  about  the  user  and  by 
amplifying  or  dilating  the  user’s  motions  -  including  both 
turning  and  walking.  These  adjustments  systematically  guide 
the  user  on  a  curved  path  within  the  real  world  while  the  user 
perceives  that  they  are  following  a  straight  line  in  the  VE. 

Appropriate  application  of  these  injected  rotations  can 
gradually  direct  users  away  from  real-world  obstacles  and 
back  into  open,  navigable  space.  If  the  rate  of  rotation 
introduced  by  RDW  is  small  enough,  it  is  not  perceived  by 
the  user  and  he  or  she  will  subconsciously  respond  by 
turning  in  the  real  world  in  order  to  maintain  the  desired 
heading  in  the  VE. 

In  (Hodgson,  Bachmann  and  Waller  2011),  the  Pis 
implemented  a  real-time  generalized  RDW  algorithm  and  demonstrated  its  use  with  live  users  for  the  first 
time  (i.e. ,  not  merely  a  computer  simulation).  This  work  included  experiments  during  which  users 
navigated  through  a  very  large  virtual  forest  and  completing  a  shopping  search  task  in  a  full-scale  virtual 
grocery  store.  During  the  course  of  these  experiments,  users  almost  always  traveled  virtual  distances 
which  would  have  placed  them  well  outside  the  physical  tracking  area  (Fig  3.).  In  spatial  memory  tasks, 
data  showed  no  detriment  to  using  the  subtle  sensorimotor  distortions  necessary  for  RDW  to  work.  Post¬ 
experiment  surveys  indicated  that  RDW  went  relatively  unnoticed  and  did  not  induce  simulator  sickness. 
In  more  recent  work  (Hodgson  and  Bachmann  2013),  the  Pis  conducted  a  systematic  comparison  of  the 
performance  of  several  alternative  generalized  RDW  walking  algorithms  and  to  establish  how  minimum 
sized  tracking  area  required  for  RDW  algorithms  of  different  type.  The  work  involved  both  live  user  studies 
and  simulations.  Users,  on  average,  were  able  to  travel  to  a  maximum  visual  distance  of  55.65m  ±  0.82 
from  the  laboratory’s  center,  despite  starting  within  22.5m  of  the  furthest  lab  wall.  The  work  established 
that  with  present  methods  RDW  could  be  carried  out  imperceptible  in  a  tracking  area  with  a  radius  of  17m 
(Fig.  4). 

As  part  of  the  previous  NSF  grant  to  develop  RDW  techniques,  the  Pis  have  recently  studied  the 
use  of  RDW  algorithms  in  constrained  environments.  A  grocery  store  (Fig.  5)  with  rigid  aisles,  for 
example,  limits  a  user’s  future  actions  to  a  small  set  of  predictable  options  and  makes  it  possible  to 
predict  a  her  or  his  likely  direction  and  distance  of  travel.  This  can  allow  RDW  to  utilize  otherwise 
undesirable  space  by,  for  example,  safely  steering  a  user  along  a  rightward  wall  when  it  is  known  that  a 
particular  portion  of  the  VE  does  not  permit  rightward  turns.  This  technique  is  dubbed  Fully  Optimized 
RDW  for  Constrained  Environments  (FORCE;  Zmuda  et  al.  2013),  and  has  compared  favorably  to 
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Fig.  3.  Typical  visual  and  physical  paths 
for  a  user  being  redirected  in  a  VE.  Small 
rectangle  represents  the  physical 
tracking  area.  Units  are  in  meters. 
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established  generalized  RDW  methods  in 
simulation.  In  failure  cases  where  generalized 
RDW  do  not  prevent  a  user  from  leaving  the 
tracking  space,  FORCE  succeeds  nearly  95%  of 
the  time. 

Finally,  the  Pis  have  demonstrated  that 
RDW  can  safely  steer  multiple  users  away  from 
each  other  when  there  are  several  immersed 
users  within  the  same  tracking  area  (Holm  2012). 

Work  is  underway  to  apply  multi-user  collision 
prediction  and  avoidance  algorithms  to  live  user 
navigation  in  a  VE. 

While  these  latter  studies  are  unrelated  to 
the  simulation  of  slope,  they  serve  to  illustrate 
diligence  in  pursuing  the  previous  objectives  that 
were  funded  by  the  NSF,  and  a  track  record  of 
successfully  applying  RDW  techniques  to 
enhance  the  safety,  utility,  and  effectiveness  of 
natural  walking  locomotion  interfaces  in  IVE. 

Portable  Position  Tracking 

Ongoing  work  by  the  Pis  has  resulted  in 
the  development  of  a  Self-Contained  Inertial 
Position  Tracking  (SCIPT)  designed  to  serve  as  a 
locomotion  interface  for  portable  immersive  VE 
systems  (Yun  et  al.  2011).  It  can  produce  smooth 
high  frequency  updates  with  drift  error  of  less  than 
1%  of  total  distance  traveled  in  outdoor 
applications,  with  occasional  GPS  fixes  correcting 
the  remaining  drift.  SCIPT  is  based  on  the  use  of 
foot  mounted  miniature  inertial/magnetic 
measurement  units  and  zero  velocity  updating, 
which  takes  advantage  of  the  fact  that  walking,  side 
stepping,  and  running  include  repeated 
recognizable  periods  during  which  the  foot  is  on  the 
ground  and  has  a  known  velocity  and  acceleration 
of  zero.  These  periods  allow  us  to  determine  and 
remove  the  sensor  drift  error  that  occurs  during 
each  step  to  accurately  calculate  the  direction  and 
magnitude  of  movement.  Nearly  all  error  in  SCIPT 
position  estimates  is  due  to  heading  errors  caused  magnetic  anomalies.  The  Pis  are  currently 
collaborating  with  the  PNI  Corporation  of  Santa  Rosa,  CA  and  have  incorporated  their  SpacePoint  9-axis 
inertial/magnetic  sensor  module  into  the  SCIPT  system.  The  SpacePoint  utilizes  a  novel  fusion  algorithm 
that  produces  superior  estimations  of  azimuth  in  the  presence  of  magnetic  anomalies.  It  is  anticipated  that 
this  collaboration  will  produce  highly  accurate  sourceless  position  estimates  in  both  indoor  and  outdoor 
environments. 

In  demonstrating  this  system  in  conjunction  with  RDW,  a  series  of  long  walks,  up  to  2km  in  length, 
were  conducted  along  a  virtual  straight  path.  As  shown  in  Fig. 6,  no  tracking  drift  was  apparent  over  these 
large  distances  and  users  were  easily  contained  in  the  outdoor  field  selected  as  a  tracking  space.  For 
indoor  applications  where  GPS  is  unavailable,  work  by  the  Pis  has  resulted  in  the  creation  of  an  entirely 
new  position  tracking  system  capable  of  providing  absolute  position  estimates  at  update  rates  similar 
GPS.  The  system  uses  TDOA  and  a  novel  Multi-Dimensional  Scaling  (MDS)  algorithm  to  resolve  the 
positions  of  all  users  in  a  common  tracking  area  (Vincent,  2012).  The  algorithm  takes  ultrasonic  range 
measurements  amongst  multiple  users  and  a  series  of  arbitrarily-placed  base  stations,  and  uses  SCIPT 
displacement  vectors  to  provide  updates  between  ultrasonic  sensor  readings.  The  range  measurements 
provide  internode  distances  between  each  immersed  user  and  between  users  and  base  nodes,  which  can 
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Fig.  4:  Maximum  physical  distance  of  simulated 
users  with  each  algorithm.  The  darker  line  shows 
the  results  when  using  a  7.5m  turn  radius;  the 
lighter  line  shows  a  22m  turn  radius. 


Fig.  5.  Predicted  path  of  a  user  in  a  constrained  VE. 
Such  predictions  can  optimize  redirection. 
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Fig.  6.  Plot  of  a  2  km  (1 .24  mi)  virtual  straight  walk 


be  resolved  to  reveal  the  global  configuration  of  users 
along  with  their  absolute  positions.  If  within  range  of  at 
least  one  other  node  in  the  system,  SCIPT  drift 
correction  can  still  be  accomplished  by  this  method. 

Thus,  the  system  can  operate  robustly  over  a  relatively 
large  area.  While  indoors,  or  when  using  RDW,  users 
should  rarely  exceed  this  distance. 

The  Pis  in  collaboration  with  researchers  at  the 
Naval  Postgraduate  School  in  Monterey,  CA.  have  also 
developed  a  system  of  locating  users  by  means  of  a 
laser  range  scanner.  The  precision,  range,  and 
resolution  of  low  cost  scanning  laser  range  finders 
makes  it  possible  to  perform  Simultaneous  Location 
and  Mapping  (SLAM)  to  both  generate  a  map  of  an  ad- 
hoc,  indoor  tracking  area  and  to  locate  users  within  that 
map.  Generating  a  room  map  has  the  added  benefit  of 
providing  physical  limit  information  to  RDW  algorithms, 
so  that  extra  failsafe  methods  can  be  employed  to 
keep  users  from  colliding  with  obstacles.  Multiple 
users  can  be  tracked  effectively  with  a  single,  tripod- 
mounted  laser  at  the  side  of  a  room.  Alternatively, 
the  laser  can  easily  be  mounted  to  the  Pis'  backpack 
rendering  system  in  an  outward-facing  configuration 
for  indoor  or  outdoor  use. 

Again,  these  prior  research  efforts  are 
somewhat  tangential  to  the  present  objective  of 
simulating  slope.  However,  we  hope  to  illustrate  the 
depth  and  experience  that  the  Pis  have  in 
developing  locomotion  interfaces  in  general,  and  in 
tracking  foot  motion  and  stepping  patterns 
specifically.  The  SCIPT  system  that  we  have 
developed  is  extremely  well  suited  to  measuring  the 
physical  effort  that  users  may  exert  when  trying  to 

navigate  up  a  virtual  hill.  Variations  in  the  acceleration  or  angle  of  one’s  foot  can  be  evaluated  when 
determining  the  corresponding  vertical  and  horizontal  displacement  in  the  virtual  world.  While  scaling 
effort  is  easily  possible  with  other  systems  -  including  treadmills  and  even  desktop  VEs  -  we  believe  that 
SCIPT  will  offer  the  highest-fidelity  experience  for  end  users. 

Significance 


Fig.  7.  Laser  ranging  scan  of  25m  x  44m  HIVE. 
Unit  is  positioned  in  the  center  of  the  wall  at  the 
bottom  of  the  plot.  Shorter  returns  indicate  the 
position  of  two  users  on  the  left. 


To  date,  most  VR  facilities  have  been  centralized,  specialized,  and  expensive,  and  have  thus 
been  relatively  unavailable  to  a  great  majority  of  the  population.  In  creating  VR  systems  that  are  portable 
and  relatively  inexpensive,  this  research  represents  a  significant  step  toward  making  VR  less  exclusive. 
De-centralizing  VR  will  make  the  technology  available  to  a  much  broader  range  of  people  than  has 
heretofore  been  possible,  providing  first-hand  exposure  to  cutting-edge  concepts  and  models  in  science 
and  technology  to  any  population  that  educators  or  researchers  chose. 

In  addition  to  representing  a  qualitative  change  in  how  VR  has  heretofore  been  generally 
conceptualized,  the  proposed  research  delivers  significant  enhancements  to  the  component  technologies 
that  comprise  it.  For  example,  while  the  commercial  XSENS  MVN  system  (Xsens  Technologies  B.V., 
2009)  is  capable  of  providing  full  body  motion  capture  and  position  tracking  without  a  supporting 
infrastructure  it  is  relatively  expensive,  and  requires  users  to  wear  complex  equipment,  and  cannot  be 
easily-integrated  into  current  VR  systems  that  simulate  large-scale  spaces.  The  ExpeditionDI  system 
(Quantum3D)  currently  in  use  by  the  US  military  is  portable  and  allows  multiple  users  to  participate 
simultaneously  in  a  virtual  simulation,  but  does  not  provide  users  with  realistic  proprioceptive  and 
vestibular  cues.  Similarly,  the  proposed  development  of  redirected  walking  in  large  physical  spaces 
successfully  enables  users  to  explore  extremely  large  simulated  spaces  in  a  manner  that  is  much  more 
realistic  than  that  provided  by  current  navigational  interfaces  such  as  the  omnidirectional  treadmill. 
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